Skip to main content
SpringerLink
Log in

Clustering Dynamics on Graphs: From Spectral Clustering to Mean Shift Through Fokker–Planck Interpolation

  • Chapter
  • First Online:
Active Particles, Volume 3

Abstract

In this work, we build a unifying framework to interpolate between density-driven and geometry-based algorithms for data clustering and, specifically, to connect the mean shift algorithm with spectral clustering at discrete and continuum levels. We seek this connection through the introduction of Fokker–Planck equations on data graphs. Besides introducing new forms of mean shift algorithms on graphs, we provide new theoretical insights on the behavior of the family of diffusion maps in the large sample limit as well as provide new connections between diffusion maps and mean shift dynamics on a fixed graph. Several numerical examples illustrate our theoretical findings and highlight the benefits of interpolating density-driven and geometry-based clustering algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. L. Ambrosio, N. Gigli, and G. Savaré. Gradient flows in metric spaces and in the space of probability measures. Lectures in Mathematics ETH Zürich. Birkhäuser Verlag, Basel, second edition, 2008.

    Google Scholar 

  2. E. Arias-Castro, D. Mason, and B. Pelletier. On the estimation of the gradient lines of a density and the consistency of the mean-shift algorithm. J. Mach. Learn. Res., 17:Paper No. 43, 28, 2016.

    Google Scholar 

  3. M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, 15(6):1373–1396, 2003.

    Article  Google Scholar 

  4. M. Belkin and P. Niyogi. Towards a theoretical foundation for Laplacian-based manifold methods. In International Conference on Computational Learning Theory, pages 486–500. Springer, 2005.

    Google Scholar 

  5. D. Burago, S. Ivanov, and Y. Kurylev. A graph discretization of the Laplace-Beltrami operator. J. Spectr. Theory, 4(4):675–714, 2014.

    Article  MathSciNet  Google Scholar 

  6. J. Calder and N. García Trillos. Improved spectral convergence rates for graph Laplacians on epsilon-graphs and k-nn graphs. arXiv preprint arXiv:1910.13476, 2019.

    Google Scholar 

  7. J. Calder, N. G. Trillos, and M. Lewicka. Lipschitz regularity of graph Laplacians on random data clouds, 2020.

    Google Scholar 

  8. M. A. Carreira-Perpiñán. Gaussian mean-shift is an EM algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(5):767–776, 2007.

    Article  Google Scholar 

  9. M. A. Carreira-Perpiñán. Clustering methods based on kernel density estimators: mean-shift algorithms. In Handbook of cluster analysis, Chapman & Hall/CRC Handb. Mod. Stat. Methods, pages 383–417. CRC Press, Boca Raton, FL, 2016.

    Google Scholar 

  10. X. Cheng and H.-T. Wu. Convergence of graph Laplacian with KNN self-tuned kernels. arXiv preprint arXiv:2011.01479, 2020.

    Google Scholar 

  11. S.-N. Chow, L. Dieci, W. Li, and H. Zhou. Entropy dissipation semi-discretization schemes for Fokker-Planck equations. J. Dynam. Differential Equations, 31(2):765–792, 2019.

    Article  MathSciNet  Google Scholar 

  12. R. R. Coifman and S. Lafon. Diffusion maps. Appl. Comput. Harmon. Anal., 21(1):5–30, 2006.

    Article  MathSciNet  Google Scholar 

  13. D. B. Dunson, H.-T. Wu, and N. Wu. Spectral convergence of graph Laplacian and heat kernel reconstruction in l ∞ from random samples, 2019.

    Google Scholar 

  14. A. Esposito, F. S. Patacchini, A. Schlichting, and D. Slepcev. Nonlocal-interaction equation on graphs: gradient flow structure and continuum limit. Arch. Ration. Mech. Anal., 240(2):699–760, 2021.

    Article  MathSciNet  Google Scholar 

  15. K. Fukunaga and L. Hostetler. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on Information Theory, 21(1):32–40, 1975.

    Article  MathSciNet  Google Scholar 

  16. N. García Trillos, M. Gerlach, M. Hein, and D. Slepčev. Error estimates for spectral convergence of the graph Laplacian on random geometric graphs toward the Laplace–Beltrami operator. Foundations of Computational Mathematics, pages 1–61, 2019.

    Google Scholar 

  17. N. García Trillos, F. Hoffmann, and B. Hosseini. Geometric structure of graph Laplacian embeddings. Journal of Machine Learning Research, 22(63):1–55, 2021.

    MathSciNet  MATH  Google Scholar 

  18. N. García Trillos and D. Slepčev. A variational approach to the consistency of spectral clustering. Applied and Computational Harmonic Analysis, 45(2):239–281, 2018.

    Article  MathSciNet  Google Scholar 

  19. E. Giné and V. Koltchinskii. Empirical graph Laplacian approximation of Laplace-Beltrami operators: large sample results. In High dimensional probability, volume 51 of IMS Lecture Notes Monogr. Ser., pages 238–259. Inst. Math. Statist., Beachwood, OH, 2006.

    Google Scholar 

  20. M. Hein, J.-Y. Audibert, and U. v. Luxburg. Graph Laplacians and their convergence on random neighborhood graphs. Journal of Machine Learning Research, 8(6), 2007.

    Google Scholar 

  21. M. Hein, J.-Y. Audibert, and U. Von Luxburg. From graphs to manifolds–weak and strong pointwise consistency of graph laplacians. In International Conference on Computational Learning Theory, pages 470–485. Springer, 2005.

    Google Scholar 

  22. V. J. Hodge and J. Austin. A survey of outlier detection methodologies. Artificial Intelligence Review, 22(2):85–126, Oct 2004.

    Article  Google Scholar 

  23. J. D. Hunter. Matplotlib: a 2d graphics environment. Comput. Sci. Eng., 9(3):90–95, 2007.

    Article  Google Scholar 

  24. E. Jones, T. Oliphant, P. Peterson, et al. SciPy: Open source scientific tools for Python, 2001–. Available at http://www.scipy.org/.

  25. W. L. G. Koontz, P. M. Narendra, and K. Fukunaga. A graph-theoretic approach to nonparametric cluster analysis. IEEE Transactions on Computers, (9):936–944, 1976.

    Article  MathSciNet  Google Scholar 

  26. A. Little, M. Maggioni, and J. M. Murphy. Path-based spectral clustering: Guarantees, robustness to outliers, and fast algorithms. Journal of Machine Learning Research, 21(6):1–66, 2020.

    MathSciNet  MATH  Google Scholar 

  27. A. Little, D. McKenzie, and J. Murphy. Balancing geometry and density: Path distances on high-dimensional data, 2020.

    Google Scholar 

  28. J. Lu. Graph approximations to the Laplacian spectra. arXiv: Differential Geometry, 2019.

    Google Scholar 

  29. J. Maas. Gradient flows of the entropy for finite Markov chains. Journal of Functional Analysis, 8(261):2250–2292, 2011.

    Article  MathSciNet  Google Scholar 

  30. L. Michel. About small eigenvalues of the Witten Laplacian. Pure and Applied Analysis, 1(2):149–206, 2019.

    Article  MathSciNet  Google Scholar 

  31. B. Nadler, S. Lafon, R. R. Coifman, and I. G. Kevrekidis. Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Applied and Computational Harmonic Analysis, 21(1):113–127, 2006. Special Issue: Diffusion Maps and Wavelets.

    Google Scholar 

  32. A. Y. Ng, M. I. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems (NIPS), pages 849–856. MIT Press, 2001.

    Google Scholar 

  33. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.

    MathSciNet  MATH  Google Scholar 

  34. G. Schiebinger, M. J. Wainwright, and B. Yu. The geometry of kernelized spectral clustering. The Annals of Statistics, 43(2):819–846, 2015.

    Article  MathSciNet  Google Scholar 

  35. J. Shi and J. Malik. Normalized cuts and image segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(8):888–905, 2000.

    Article  Google Scholar 

  36. Z. Shi. Convergence of Laplacian spectra from random samples. arXiv preprint arXiv:1507.00151, 2015.

    Google Scholar 

  37. A. Singer. From graph to manifold Laplacian: The convergence rate. Applied and Computational Harmonic Analysis, 21(1):128–134, 2006.

    Article  MathSciNet  Google Scholar 

  38. R. Tibshirani, G. Walther, and T. Hastie. Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(2):411–423, 2001.

    Article  MathSciNet  Google Scholar 

  39. H. tieng Wu and N. Wu. When locally linear embedding hits boundary, 2018.

    Google Scholar 

  40. D. Ting, L. Huang, and M. Jordan. An analysis of the convergence of graph laplacians. arXiv preprint arXiv:1101.5435, 2011.

    Google Scholar 

  41. L. van der Maaten, E. Postma, and H. van den Herik. Dimensionality reduction: A comparative review. Tilburg University Technical Report, TiCC-TR2009-005. 2009.

    Google Scholar 

  42. S. van der Walt, C. Colbert, and G. Varoquaux. The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng., 13(2):22–30, 2011.

    Article  Google Scholar 

  43. R. Vaughn, T. Berry, and H. Antil. Diffusion maps for embedded manifolds with boundary with applications to PDEs, 2019.

    Google Scholar 

  44. A. Vedaldi and S. Soatto. Quick shift and kernel methods for mode seeking. In European conference on computer vision, pages 705–718. Springer, 2008.

    Google Scholar 

  45. U. von Luxburg. A tutorial on spectral clustering. Statistics and Computing, 17(4):395–416, Dec 2007.

    Article  MathSciNet  Google Scholar 

  46. U. von Luxburg, M. Belkin, and O. Bousquet. Consistency of spectral clustering. Ann. Statist., 36(2):555–586, 2008.

    MathSciNet  MATH  Google Scholar 

  47. E. Witten. Supersymmetry and Morse theory. Journal of Differential Geometry, 17(4):661–692, 1982.

    Article  MathSciNet  Google Scholar 

  48. C. L. Wormell and S. Reich. Spectral convergence of diffusion maps: improved error bounds and an alternative normalisation, 2020.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Eric Carlen, Paul Atzberger, Anna Little, James Murphy, and Daniel McKenzie for helpful discussions. KC was supported by NSF DMS grant 1811012 and a Hellman Faculty Fellowship. NGT was supported by NSF-DMS grant 2005797. DS was supported by NSF grant DMS 1814991. NGT would also like to thank the IFDS at UW-Madison and NSF through TRIPODS grant 2023239 for their support. The authors gratefully acknowledge the support from the Simons Center for Theory of Computing, at which this work was completed.

Author information

Authors and Affiliations

  1. University of California, Santa Barbara, CA, USA

    Katy Craig

  2. University of Wisconsin, Madison, WI, USA

    Nicolas GarciaTrillos

  3. Carnegie Mellon University, Pittsburg, PA, USA

    Dejan Slepčev

Authors
  1. Katy Craig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicolas GarciaTrillos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dejan Slepčev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dejan Slepčev .

Editor information

Editors and Affiliations

  1. Department of Mathematical Sciences, Politecnico di Torino, Turin, Italy

    Nicola Bellomo

  2. Mathematical Institute, University of Oxford, Oxford, UK

    José Antonio Carrillo

  3. Department of Mathematics and Institute for Physical Science & Technology, University of Maryland, College Park, MD, USA

    Eitan Tadmor

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Craig, K., GarciaTrillos, N., Slepčev, D. (2022). Clustering Dynamics on Graphs: From Spectral Clustering to Mean Shift Through Fokker–Planck Interpolation. In: Bellomo, N., Carrillo, J.A., Tadmor, E. (eds) Active Particles, Volume 3. Modeling and Simulation in Science, Engineering and Technology. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-93302-9_4

Download citation

Publish with us

Policies and ethics

Search

Navigation